The present invention relates to a high-quality portrait image forming apparatus for obtaining a more natural portrait image from a portrait image and, more particularly, to a red-eye detection/retouch apparatus for correcting a portrait image including red pupils photographed in use of a flashbulb to a portrait image having a more natural pupil color.
When the periphery of a target object is dark, and a person is to be photographed with a camera using a flashbulb, the pupil portions of eyes are often photographed as red eyes. When the periphery is dark, the pupil of the eye is open. The flashlight is reflected by the retina through the pupil and reaches the camera, resulting in such an unnatural image. Particularly, along with recent size reduction of cameras, the optical axis of the photographing lens approaches that of the flashbulb, and the red-eye phenomenon tends to easily occur.
In a "prevention of red-eye effect at flashlight photography" disclosed in Japanese Patent Laid-Open No. 52-80120, only the flashbulb is caused to emit light in advance immediately before the photographing operation to contract the pupils, thereby preventing the red-eye phenomenon in the photographing operation. However, this method may give a sense of incompatibility to the object person or largely consume the battery.
In an "electric camera" disclosed in Japanese Patent Laid-Open No. 4-192681, flashlight photographing and normal photographing are continuously performed to detect and remove a reflection component due to the flashlight. However, when the object person is moving, a positional shift may be generated between the two photographed images, or the reflection component cannot be effectively detected because the normally photographed image is dark.
In an "image processor" disclosed in Japanese Patent Laid-Open No. 5-224271 which uses information representing the presence/absence of use of a flashbulb, the focal length information of the photographing lens, distance information between the photographing lens and the electronic flash light-emitting portion in the photographing operation, these pieces of information must be acquired and transmitted to the image processing unit in the photographing operation. However, a camera having such a function is not general and therefore is not effective. Accordingly, red-eye detection/retouch apparatuses for detecting photographed red eyes and correcting the red eyes to a natural color tone of the eyes are conventionally proposed and used.
FIG. 2 shows an input image signal 1 including a red eye as an example of an input image. On this input image, assume that the pupil at the central portion of each eye is red. Two examples of a conventional "pink-eye effect position detector" disclosed in Japanese Patent Laid-Open No. 6-258732 will be described below.
As the first example, an apparatus having the following function is described in the above prior art. The color image of a color film which has undergone development is picked up to obtain a film image. At least one of the low chroma part and the low illuminance part of the film image is extracted, thereby extracting an area including a black eye. In addition, an area including the face is extracted by extracting a skin color part from the film image. Similarly, a pink-eye area is extracted by extracting a red color part from the film image. The pink-eye area is automatically detected under a condition that the pink-eye area (pink eye) exists in the black area (pupil) in the skin color part (face). In the embodiment of this prior art, the unit for detecting the face and pupil area is realized by threshold processing, and the unit for detecting the pink-eye area in the pupil in the face is realized by a combination of area expansion and AND. An embodiment in which the area is selected considering that the face or eye has an almost elliptical shape is also described in this prior art.
As the second example, an apparatus having the following function is described in the prior art. As in the above-described conventional apparatus, the user designates a point in the pink-eye area of a color film image by using a coordinate designation unit such as a touch panel, thereby acquiring color information of the designated point. Pink-eye area detection is realized by extracting an area having a color close to the color information by threshold processing.
Two examples of a conventional red-eye detection/retouch apparatus based on a "pink-eye effect retouch system" disclosed in Japanese Patent Laid-Open No. 6-350914 will be described below.
As the first example, an apparatus having the following function is described in this prior art. The image of a negative color film which has undergone development is converted into a film image by a scanner. Thereafter, colors in the film image are subjected to threshold processing, thereby detecting a pink-eye area. After the contour line of the detected area is extracted, the density distribution of pixels on an x axis (horizontal line) passing the center of gravity (x.sub.0,Y.sub.0) of the contour line is checked, as shown in FIG. 3A. When x1 and x4 corresponding to an iris edge 2 are detected, as shown in FIG. 3B, a pink-eye area is finally determined, thereby realizing pink-eye area detection. Reference numeral 3 denotes a pupil as a pink-eye portion.
This conventional apparatus performs pink-eye detection processing for the entire frame. The performance of pink-eye detection processing is poor, and detection often fails. For this reason, the original pink-eye area may not be detected, or an area other than the pink-eye area may be detected as a pink-eye area. To solve this problem, the second example having the following function is described in the prior art. The user designates a small area including a pink eye shown in FIG. 4 using a touch panel or the like as a pink-eye designation area 4. The above processing is performed only for the pink-eye designation area. With this arrangement, an apparatus for preventing detection failures is realized without increasing the work load to the user.
In a "red-eye effect correcting device" disclosed in Japanese Patent Laid-Open No. 7-13274 as well, the user designates an area where two red eyes exist, as shown in FIG. 5, using a touch panel or the like. Thereafter, threshold processing is performed on the basis of the average chromaticity of the area, thus realizing recd-eye area detection.
The conventional processing of automatically detecting the pink- or red-eye area uses threshold processing based on colors or the shape of an area which has been detected by the threshold processing. A technique applied to the entire frame is not practical because of its low detection accuracy, as is known. On the other hand, the method of causing the user to designate a point in the red-eye area increases the work load to the user in designation because the red-eye area is very small on the image. The compromise idea of these two methods, i.e., the method of causing the user to designate a small area including red eyes is effective because the work load to the user is small, and detection failures can be suppressed. In this conventional method, however, red-eye detection processing for the entire frame is only simply applied to the specific area, and the shape information of the area given by the user or position information of the detected red-eye area is not positively used. For this reason, it can hardly be said that detection errors can be sufficiently reduced, and a further improvement is required.